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INTRODUCTION 

A  situation  which  sometimes  arises  is  that 
in  which  a  time  series  y(t),  t  ■  0,1,...  is  to  be 
modelled  as  a  function  of  another  time  series  x(t), 
t  •  o,l,...  (or  more  generally  as  a  function  of  k  series 
x1(t),...,xk(t);  t  «  0,1,...).  It  Is  often  the  case 
that  for  any  time  t,  y(t)  will  depend  not  only  on  x(t), 
but  also  on  x(t  -  1),  x(t  -  2 ),..., x(t  -  r)  for  some  in¬ 
teger  r  ^  0,  and  sometimes  on  the  complete  past  history 
of  x(t).  When  all  of  the  series  are  second  order  sta¬ 
tionary  and  a  large  number  of  observations  Is  available, 
the  powerful  methods  of  spectral  analysis  may  be  employ¬ 
ed  to  estimate  various  types  of  linear  models.  On  the 
other  hand,  when  relatively  few  observations  are  avail¬ 
able  and  the  series  Involved  may  be  nonstationary ,  at 
least  to  the  extent  of  having  trending  means,  these 
methods  may  not  be  appropriate,  and  yet  it  may  be  desir¬ 
able  to  have  some  sort  of  frequency  decomposition  of  the 
series.  (For  instance,  it  may  be  of  interest  to  model 
trends  and  stationary  parts  of  the  series  separately.) 
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? 

The  purpose  of  this  study  is  to  investigate 
some  of  the  properties  of  a  method  whereby  y(t)  is 
modelled  not  directly  as  a  linear  function  of  x(t), 
x(t  -  1 x(t  -  r) ,  but  rather  as  a  linear  function 
of  some  of  the  principal  components  of  x(t),x(t  -  1), 

. ,.,x(t  -  r).  Chapter  I,  formulated  in  terms  of  a  set 
of  random  variables  y,x^,...,Xp,  investigates  the  co- 
variance  structure  among  these  variables  which  will 
yield  the  result  that  the  first  m  <  p  principal  com¬ 
ponents  of  x^.^.Xp  provide  a  better  predictor  of  y 

than  does  any  subset  x4  . 

X1  xm 

In  chapter  II ,  principal  component  processes 
of  x(t),x(t  -  l),...,x{t  -  n)  are  defined  and  their  pro¬ 
perties  noted.  It  is  shown  that  when  x(t)  is  one  of  the 
usual  types  of  second  order  stationary  processes,  the 
principal  component  processes  effect  a  partial  frequency 
decomposition  of  x(t). 

In  chapter  III,  It  Is  shewn  that  for  certain 
processes  containing  deterministic  components,  certain 
of  the  principal  component  processes  tend  to  filter  out 
these  deterministic  components. 
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An  example  of  the  behavior  of  the  principal 
component  processes  of  an  artificially  generated  process 
Is  shown  In  the  appendix  together  with  the  principal 
components  of  an  autoregressive  process*  some  of  the 
transfer  functions  corresponding  to  the  eigenvectors 
of  a  moving  average  process  *  and  the  first  two  principal 
component  processes  derived  from  a  series  representing 
quarterly  total  U,S.  personal  income. 


CHAPTER  I 


REGRESSION  ON  PRINCIPAL  COMPONENTS 
Let  X  be  the  p  component  random  vector 


*  * 
X 


1XPI 


with  covariance  matrix  E.  It  will  be  assumed  that  EX  ■  0, 
since  the  mean  vector  does  not  enter  into  the  following 
discussion.  There  exists  an  orthogonal  linear  transforma¬ 
tion 

z  »  **x 

such  that  the  covariance  matrix  of  Z, 


E(ZZ')  *  A  * 


where  x,  >...>  xp  are  the  roots  (eigenvalues)  of  the 
characteristic  equation  |£  -  xl|  ■  0,  and  *  * 
with  the  eigenvector  corresponding  to  Xj .  The  rth 
component  of  Z,  Z *  $r'X  is  called  the  rth  principal 
component  of  X  and  has  maximum  variance  of  all  normalized 


<*r’  ♦r 


1)  linear  combinations  uncorrelated  with 


Zl*****Zr  1*  In  Partlcular  zi  the  first  principal 
component  has  maximum  variance  of  any  normalized  linear 


combination. 
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i-loreover , 


P  ->  P  ?  , 

E  E  ( Z  ) 2  -  E  -  t  r!.  -  trE  -  E  E(x,)2  (1.0) 

i-1  1  1-1  1  1-1 


Principal  components  have  found  application 
especially  in  exploratory  studies  where  the  number  p 
of  variables  under  consideration  may  be  large.  It  may 
happen  that  a  few  of  the  principal  components  explain 
most  of  the  variation  (as  shown  by  (1.0))  of  the  ori¬ 
ginal  p  variables.  Another  criterion  often  applied  to 
the  use  of  principal  components  is  that  they  should  have 
some  sort  of  "reasonable"  interpretation  in  their  own 


right. 


When  the  variables  x.,...,x  are  to  be  used  as 
possible  explanatory  variables  in  a  linear  model,  the 
use  of  principal  components  gives  the  additional  advan¬ 
tage  that  the  Z^'s  are  mutually  uncorrelated.  A  discuss 
ion  of  the  use  of  principal  components  in  regression 
analysis  is  given  in  [  8  ]  along  with  some  examples  deal 
ing  with  economic  modelling. 

The  vectors  form  a  basis  for  the  p 

dimensional  Euclidean  space  Hp.  If  ^,...,1^  is  any 
other  orthonormal  basis  for  ,  it  can  be  shown 
[10  p.  400]  that  for  any  k  <_  p, 
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P  k  2  p  k  2 

E{  1  |  x  -  rZ®|}<E{Elx-EU*!), 

1*1  i  v-i  v  lv  “  1*1  i  \>-l  V  iv 


where  * 


[♦ill 

• 

*  *  * 

♦il 

• 

• 

i 

m 

% 

%  * 

np 

(1.1) 


and  U  *  *  'X.  or 
v  v 


that  the  k  components  Z^ , . give  a  better  approxi¬ 

mation  in  the  sense  of  expected  squared  error  to  x^(...tx 

than  do  the  k  coefficients  of  any  other  orthogonal  basis. 

P  . 

Moreover,  when  X  is  scaled  so  that  l  E(x,?)  »  le 

i-1  1 

then. 


k  k 

-  I  X  log  X  <  -  l  o  log  o  , 
i-1  1  1  “  i-1  1  1 


(1.2) 


2 

where  ).  Equality  holds  in  (1.2)  only  when 

k  k 

l  X,  »  l  o  for  every  0  <  k  <  p.  (1.1)  and  (1.2) 
1-1  1  i-1  1 

show  that  for  any  fixed  k  <  p,  the  first  k  principal 
components  carry  at  least  as  much  of  the  information 
contained  in  the  set  x  ,...,x  of  variables  as  any  k 
linear  combinations,  including  any  subset  x.,...,x.  . 


When  x x  .ire  to  be  com;  Ide-ed  as  cosstbie  explar.a- 
tory  variables  in  a  linear  rode!  for  another  variable, 
say,  y,  then  it  is  of  interest  to  be  able  to  characterize 
when  the  prinrlpa’.  components  provide  more  in¬ 

formation  relevant  to  y  than  some  other  linear  combina¬ 
tions 

Let  y,  X'  *  (x.,..,x  )  be  given  random  variables 

■*  H 


such  that  Ey  *  ECx^  -  0,  1*1, EXX'  ■  E ,  EyX'  * 

E ^  ■  E2i.  As  before  let  >_  Xp  and  4^,..., 

be  the  eigenvalues  and  corresponding  orthonorraalized 


eigenvectors  of  E .  Let  denote  the  vector 


0 


» 


• 

0 

1 

0 


ith  row 


Ul 

and  3  be  the  regression  vector  of  y  on  X  (i.e.  the  vec¬ 
tor  minimizing  the  expression  E(y  -  X'B)^.)  Suppose  now 
that  y  is  to  be  predicted  by  a  linear  combination  of 
k  <  p  random  variables  which  are  themselves  linear  combin¬ 
ations  of  x  ,...,x  .  One  possibility  is  to  choose  some 

subset  x,  . x.,  ,  while  another  is  to  choose  the  first, 

11  k 


k  principal  components  Z. 

—  K 
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Let  Z  - 


X,  where  *  (i^, a nd 


let  W 


E(k)*X»  where  E(k' 


(e  *  > • • » e j  )• 
il  xk 


Let  y  *  Z  a  denote  the  predictor  of  y  based  on  Zt 


where  a  - 


/  A 


is  the  vector  minimizing  E(y 


Z’a)2. 


A  |  A. 

and  let  y^  ■  V  c  be  the  predictor  of  y  based  on  W, 
where  c  is  the  vector  minimizing:  E(y  -  W  c)z.  Then 

yz  «  X  'a  »  X  <f,  while  y,rf  *  X  ElK,c  -  X >. 

Definition:  Let  u  be  a  vector  in  R_  and  define  the 

"  . .  P 

norm  j|u||  by  |ju||2  »  (ulu),  where  (u,v)  *  u*tv. 
Lemma  (1.1):  E(y  -  yz)2<  E(y  -  y w)2  iff  ||B  -  *|| 


< 


II  8  -f |  | 


a 


Proof:  E(y  -  9,,)''  ■  E(y  -  X’<P)"  »  E{(y  -  X*8) 

+  X’(8  -  -P)}2  =  E(y  -  X'S'I2  »  E[(B  -  ♦ 


!i  ( y  -  X  1  ?  ' 2  *  |  |  8  -  ♦  | 


Similarly,  E(y-yw)2 


E(y  -  X'B)'5  +  |j8  -  from  which  the  lemma 


follows . 


Theorem  1.1;  E(y  -  yz>?<  E(y  -  yw)?  iff 

1  ^  i  i  - 

Zl2(E  "  i-l  Vi^i,)E21  «  2 12  (E”1  -  ^k^21»  where 


*  I’ll?  ^l^k 

0...O  1  0  . .  ,  O...0O...O 


*  ^2^2  ^2^k 

0  »  •  •  CJ  0  ■  •  •  CT  0  «  a  a  O  0  •  a  •  0  f 


ij,1!  lkl2  Vk 

0  ...  o  K  x  0  ...  o  K  0  ...  c  0  ...  0 


Vk  , 

and  where  a  is  the  jkth  element  of  S^”1  where 


Vi  • 


Vk 


•  •  • 


r 


in 


* 

(3  -  $.e 

-  *) 

-  (6,8)  - 

2(8 

A 

(8, 

,6)  =  l1?Z~ 

1e21‘ 

,  (B,*)  * 

6'E* 

A 

a  * 

£12  £‘li;i 

<k)a 

*  Z 

12v 

A 

a. 

a  *  [E(ZZ')]"1  E(Zy)  « 
[E(0(kJ,XX'*(k,)3“1  EU(k)*Xy) 

-  [*<k>'l *wrl  4>Ck)*  *21 


<£♦ 


l...E4>k)] 


-1 


♦  ^’E 


21 


A.  0 

0  X, 


-1 


<K(k),Z21  -  A“Vk),l21f 


*  Mf'  -]  fitV 

so  that,  (8,*)  -  £12*  A  *  E21- 

“  fir  V  (k)  * 

(♦,*)  *  4TA  =  a'  (t)v  ;  a 


$(k)  A”1#* k* '  Z(J>(k)  A“1«(k) '  jr.1 
12  21 


Z,^(,<JA-lAA-^(k),E21  -  E120(k)A-^(^’E 


’12 


‘21* 


I 


1 


Hence,  ||fl  -  4>  |  |  ?  =  -  J\  ?  k )  A_  k }  *  z?1 


21 


Z12Z  1z21  "  S 12 C . TT*l*i,)E 
l“l  i 

'  r«trl  '  jj-TT 

Similarly,  ||8  -  V|[2  -  (B,8)  -  2(S,t)  +  (¥,  T) . 

A  *  ( k )  *  *  *i 

C 6  » ’l' )  *  3*  IE  cf  where  c  =*  [E(WW  )  ]~iF.(Wy ) 
-  [E(E(k)'xX'E<k))]'1  E(E(k>'xy) 


-  [E<k>'lE<k)r1  E(k)'i 


21 


-1 

(rej  ...Eej  )]  El  ;  E?1 


'1  i 

Ik 


cl  l  •  •  • • 1 

Vl  1k  k 


-1 


F(k),r  »s-1E(k),E 

21  Sk  E  E?l* 


Thus,  ( 3  ,*)  -  I12z"1EE(k'^-1E(k),I21. 

»  E12E(k)Sk-1E(k),E?1,  while  Cf.V)  = 


>1  *  £  >v 
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I  E(k^S  5 '  EF.^^S  “1E^k^E 

12  k  k  21 


( k)  i  _1  / 1,)  t 

■  V  s“  SA  E< 


r(k).  -l-(k)' 


*21 ' 


Hence  j ( 6  -  y | 


r  r“*  If  r  p  (k  )  «  ■*l*pCk)*p 

Z12Z  "21  *  L12E  Sk  E  1 


21 


ri2[E*’1  ‘  ^]Z 


k J  21* 


and  the  theorem  follows  from  lemma  1.1. 

In  practice  if  x^,...,x  are  highly  inter- 

k 

correlated,  then  it  might  be  expected  that  Z  -4—  ♦  ♦. » 

i-i  i  11 

would  more  closely  approximate  I”1  then  would  If  at 

the  same  time,  y  is  essentially  equally  correlated  with 
all  or  many  of  the  Xj's,  then  it  would  seem  likely  that 
the  inequality  would  hold.  In  particular,  this  situation 
might  arise  when  x1,...,x^  are  elements  of  a  time  series 
x(t),  x(t  -  l),...,x(t  -  p  +  1)  and  y  *  y(t). 

As  an  example,  suppose  x(t)  is  a  first  order 
autoregressive  process  with  x(t)  -  .9x(t  -  1)  *  Z(t) 

(a  white  noise  process).  Then  the  covariance  matrix  Z 


x( t  - 

1). 

x( t  - 

2)  can  be  taken  as 

1.0 

.9 

.81 

* 

.9 

1.0 

.9 

,  with  eigenvalues  X.  * 

.81 

.9 

1.0 

L 

2.71*, 


.19,  * 


07  and  eigenvectors 


HOO 


13 


.571 

'  -  .707  ' 

B 

.590 

»  ♦?  " 

0.0 

.571 

d 

70  7 

1  1  w  1 

* 

-.417 

.807 

-.417 


5.232  -4.687 
4.687  9.431 
■  .027  -4.68? 


0  0 
0  0 
0  0 


.117 

.121 

.121 

.125 

.117 

.121 

• 

Letting 

♦iV 

)  z  •  . 

1  21 

21  * 

.071. 

.027  ] 
4.687 
5.23?  ) 

.117  I 
.121 
.117  J 


.  Letting  k  »  1, 


,  and 


.99  .98), 


017  while 


CHAPTER  II 

STOCHASTIC  PROCESSES  AND  REPRESENTATIONS 

The  type  of  stochastic  process  x(t)  to  be  con¬ 
sidered  will  be  a  family  of  random  variables  with  index 
set  T,  where  unless  otherwise  stated  T  will  denote  a 
countable  set  of  the  real  line,  so  that  x(t)  is  a  dis¬ 
crete  process.  The  random  variables  x(t)  may  be  real 
or  complex  valued,  but  will  always  be  assumed  to  satisfy 
the  condition  E|x(t) |2<  «  for  all  t  e  T.  The  mean  value 
function  of  x(t)  will  be  denoted  by  u(t),  and  unless 
otherwise  stated  it  will  be  assumed  that  u(t)  =  0.  The 
covariance  function  will  be  denoted  by  r(s,t) 

■  E(x(s)  xTt?}  -rhere  the  bar  denotes  complex  conjugate. 

Of  particular  importance  is  the  case  where  x(t)  is  a 
second  order  stationary  process.  In  that  case  the  covari¬ 
ance  function  has  the  form 

E{ x( s )  x(s-t)}  -  r(t), 

and  r(t)  has  the  spectral  representation 

1* 
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r(t) 


n 

/  e 

_  rr 


itw 


dF( u)  , 


where  F(w),  the  spectral  distribution  function  of  x(t), 
is  real  valued,  nondecreasing,  bounded,  and  can  be 
chosen  so  that  P(-«)  *  0,  F(+*)  ■  r(0),  and  F(w)  is 
continuous  to  the  right.  If  P(w)  is  absolutely  con¬ 
tinuous  then  f(w)  ■>  F'  (w)  is  the  spectral  density  func¬ 
tion  of  x(t).  In  addition,  if  x(t)  is  second  order 
stationary.  It  has  the  spectral  representation 

n  1t. 

x ( t )  -  ;  eltw  de(»), 

-n 

whei*e  £(u>)  is  a  process  of  orthogonal  increments  which  can 
be  chosen  so  that  E  £(u>)  «  0,  E|£(w)|^  ■  F(w),  and 
E|d£(w)|2  *  dF(u).  (£(w)  is  called  the  spectral  process 
associated  with  x(t).)  From  this  spectral  representation 
the  process  x(t)  can  be  thought  of  as  being  built  up  of 
mutually  orthogonal  elementary  harmonic  oscillations 
eitu  d£(u>)  . 

Let  x(t)  be  a  second  order  stationary  process  and 
consider  a  process  y(t)  formed  from  x(t)  by  a  linear 
transformation  of  the  type 
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y(t)  =  z  b  x ( t  -  tv).  (2.1  > 

v*l 

Then 

n  » 

y(t)  *  I  b  /  ei(  t-tv)«d£ 

v«l  -«  x 

*4*.,.  n  -it„w 

-  /  elU)  (  E  bv  e  v  )  d  C  (■). 

*00  V*1  ^ 

Thus,  y (t )  is  a  second  order  stationary  process  with  ele¬ 
mentary  harmonic  oscillations  h(w)  d  £x(w),  where  h(u>) 
n  -itvw 

*  E  b  e  .  The  covariance  function  r  (t)  of  y(t, ) 
v«i  v  y 

has  the  representation 

m 

ry(t)  -  f  eitt0  |h(w)|2  d  Px(«). 

If  x ( t )  has  spectral  density  function  fx(w)  then  y(t) 

has  spectral  density  function  f  (w)  -  } h ( ui>  | 2  f  (»). 

y  x 

Thu3  for  a  given  frequency  &> ,  f^u)  is  multiplied  by 

2 

the  amount| h(u) |  to  give  fy(o)),  This  may  have  the 
effect  of  greatly  repressing  certain  frequencies  while 
enhancing  others.  For  this  reason  the  linear  operation 
(2.1)  may  be  thought  of  as  a  filter  applied  to  the  process 
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x(t).  !h(w)|‘?  is  called  the  transfer  function  of  the 

filter  and  h(<u)  the  gain  of  the  filter. 

Karhunen  -  Lofcve  Representations 
Theorem  [see  7  p.  ^78] 

Let  x(t)  be  a  stochastic  process  continuous  in  quadra¬ 
tic  mean  and  defined  for  t  e  T  a  closed  Interval.  Then 
x(t)  has  an  orthogonal  representation  of  the  form 

o» 

x(t)  -  Z  *n(t)  Zn, 
n*l 

where  *  (t)  is  an  eigenfunction  of  the  covariance  func- 
n 

tion  R{ t  ,t ' )  -  E{x(t)  x(t*)).  That  is 

/_  R(t,f)  *  (t*)  dt’  -  X  *  (t), 
i  11  n  n 

and 

\pn  C  t )  dt  *  6mn.  The  random  variables  Zn 

are  given  by 

Zn  "  ;T  x(t)  *n(t)  dt» 

and  the  Z  's  have  the  property  that  E(Zm  1  )  =  X  6 

n  m  n  n  mn 

Vfhen  x(t)  is  a  finite  discrete  process,  T  can  be  taken 
as  the  set  T  *  {1,2,...,N},  and  R(t,t')  becomes  the 
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N  X  N  isatrix 


(N) 

R 


R(l,l)  ....  R(1,N) 
R(2,l)  ....  R(2,N) 


^  R(N,1)  ....  R(N,N)J 


with  eigenvalues  ~  •••  i  and  corresponding 

orthonormal  set  of  eigenvectors  ♦!»... ,$N,  where 


*k(lV 


[V"> 

becomes 


x(t)  * 


.  The  representation  for  xCt) 


n-1  n 


then 


where 

N 

Z  ■  E  x(t)  i  (t) . 
n  t-1  n 

Thus  in  this  case  is  the  nth  principal  component  of 
the  random  variables  x(l ) , . . . ,x(N) . 


Principal  Component  Processes 
Let  x(t),  t  »  1,2,...  be  a  discrete  stochastic 

process . 

Definition:  x(t)  will  be  called  covariance  stationary  if 

E{(x(t)  -  y(t))  (TT57  ~  pts))}  •»  r(t  -  s),  where 
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u(t)  •  E  x(t).  (w(t)  is  not  necessarily  constant.)  For 

any  fixed  positive  integer  n  let  be  the  covariance 

matrix 


R 


(n) 


f  r(0)  r(l)  ....  r(n) 
r(«l)  r(0).  .  .  .  r(n-l) 


r(-n)  r(-n+l)  .  .  r(0) 


>_  Xx(n>  >  X2«n)  i  ^  be  fch®  eigenvalues  of 

and  be  the  corresponding  set 

of  orthononsalised  eigenvectors. 

Definition:  The  processes  (t);  J  ■  0,1,..., n  where 

Z.<n)(t)  -  l  ♦  (n)  (v)  x(t-v) 

J  v»0  " 


will  be  called  the  principal  component  processes  of  x(t). 

From  this  definition  and  the  usual  properties  of 
principal  components  it  follows  that: 


(1}  Z0(t)  is  that  linear  combination 


n 

I  a4_  x(t-l)  with  maximum  variance  sub- 
i»0  10 

n  2 

Ject  to  the  restriction  that  l  a,.  ■  1# 

i-1  10 

Z^n^(t)  is  the  linear  combination 
n 

Z  a.,  x(t-l)  with  maximum  variance 
i»l  11 
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5 

subject  to  the  restrictions  that  E  a.,  ■  1 

1»1  11 

n  _ 

and  E  a.^a..-.  ■  0,  and  so  forth  for 
i-1  iU 

ZJ<R)(t>»  5  “  °i‘*‘*n* 

(ii)  B{Zi(n>(t)  Zjln,"(t))  -  fijj. 

(ill)  x(t)  has  the  representation, 

x(t)  -  E  Z  (n)(t). 

u-0 

This  representation  differs  in  general  from 
the  Karhunen-Lofeve  representation  or  the 
spectral  representation  in  that  the  index 
t  is  contained  In  the  random  element.  Thus 
the  representation  is  in  terms  of  constants 
and  random  processes  rather  than  random 
variables  and  deterministic  functions  of  t. 

n  ? 

(iv)  r(0)  -  l  U  (0) |  X  . 

v-0  v 

An  interpretation  of  these  processes  can  be  seen 
more  readily  in  the  case  that  x(t)  is  a  second  order 


?1 


stationary  process.  Let  x(t)  be  r  discrete  stationary 
process  with  spectral  process  £^(u>),  and  assume  more¬ 
over,  that  x(t)  has  a  spectral  density  function  fx(w). 
Then 


Z.(n>(t)  ■  l  ♦<n)<v)  x(t-v),  J  ■  0,...,n 
J  v»0  ^ 


is  a  linear  operation  on  x(t)  and  hence,  it  follows 
that, 

(v)  Z,(n)(t)  is  a  stationary  process. 


(vi)  Zj^(t)  is  built  up  of  elementary  harmonic 

oscillations  of  the  form  eit(il  (w)d£x(w) 

w  (n)  .  x  v  -x"-u’  i  (n)  . 

where  gv  (u>)  ■  :  c  *v  (k), 

k»0 

(n) 

(vii)  Zj  (t)  has  spectral  density  function 


f7  (n)(w)  »  |ps  ^n^<o>)|  2  f  («). 

J  x 

Prom  (iii),  (vi),  (vii)  it  follows  that  the  spectral 
representations  for  x(t)  and  r(0 )  may  be  written  in  the 


forms 


x(t )  «•  /  e*Cu  d£  («) 

-n  *■ 

n  _ 

*  /  e  w(  6)  p,n(u»)  d£  (u)  +... 
-n  u  u  x 


+  *n(0)  Kn(w)  dCx(w)  ), 

and 


n 

r(0)  ■  /  f  (u)  dm 

-n  x 

•I  ( 1 ( 0 ) | 2  |g  (u)|2  fx(u)  +... 

-n  0  o 

♦  Un(0)!2  lKn(“)|2  fx(u))du» 
which  gives  a  decomposition  for  the  elementary  harmonic 


oscillations  eitu  d£x(u>)  and  the  spectral  density  func¬ 
tion  fx(u).  Prom  the  properties  (i)  and  (li)  it  follows 
that 


n  5  ^2 

/  |g  (w)|  f  (u>)  du  *  u>-  >  /  |h(u)|  f  (u)  du, 

-n  0  0  “  -n  x 

for  any  h(w)  satisfying 


n  n  p 

h(w)  -  l  bv  e"lvt),  I  bv  ■  1. 
v*0  v*0 
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Thus,  the  filter  function  gg(u>)  passes  the  most  Import¬ 
ant  portion  of  the  spectrum  of  x(t),  while  g^(iu) , . . .  ,gn(w) 
pass  increasingly  smaller  portions  of  fx(w).  The  exact 
properties  of  these  filter  functions  are  determined  by 
the  filter  coefficients 

$v(k),  k  *  0,...,n;  v*0,...,n 

which  in  turn  are  determined  by  the  covariance  structure 
of  the  process  x(t).  These  properties  are  further  illus¬ 
trated  by  the  following  special  case. 

x ( t )  A  Periodic  Process 

Let  x(t)  be  a  stationary  process  such  that 
x(t  ♦  N)  ■  x(t)  for  some  integer  N.  This  process  can  be 
thought  of  as  containing  only  a  finite  number  of  elemen¬ 
tary  harmonic  oscillations  and,  hence,  has  the  represen¬ 
tation 


N-i  .,211.1s 

x(t)  -  eU-j-’t 


(2.2) 


J-0 


where  the 


1  N-l  . 

X.  *  N  1  e  *  N  x(k),  j  -  0 . 


k»0 


n-l  are 


2*1 


mutually  orthogonal  random  variables.  (2.2)  can  be  put  In 
the  more  usual  form  by  letting 


Then  (2,2)  becomes 

N-l  1»  t 

x(t)  «  z  e  J  x ,,  -n  <  to  <  n. 
j-o  J  3 


Prom  the  definition  of  x(t),  E  x(t)  x(t  -  (M  -  j)) 

-  E  x(t)  x(t  +  J>  or  r(n  -  J)  •  r(-J).  Hence,  the 


covariance  matrix 


(N-l) 

R 


'  r(0)  r( 1)  ...  r(N-l) 
r(-l)  r(0) . . .  r(N-2) 

a 

r(-H+l)  r(-H*2) ...r(0) 


r(0)  r(  1)  ...  rU-l)l 
r(N-l)  r(0 ) . .  r(N-2)J 


[r(l)  r(2 ) .  . . .  r(0)  J 

is  a  circular  matrix.  As  such  It  has  eigenvalues  given  by 


N-l 


X  »  l  &.K  r(k) ,  where  p  is  an  Nth  root  of 
J  k«0  J  3 


unity.  The  corresponding  eigenvector  is  given  by 
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I 

I 


(n) 

♦ 

J 


1 

In¬ 


putting  0 

J 


gives 


♦ 


J 


(n) 


(2.3) 


(n) 


The  principal  component  process  Zj  (t)  then  has  the  form 

i2njv 

„  ,  r. 

J 


Z,(n)(t)  «  Ze  W  x(t-v)  J*" 


1  II-l  ^2TIJ V  12nk(t-v) 


v* 


E  e 
0 


z 

k*0 


r 


X,. 


1  N-i  I2r(  .1~k)  v  i2Hkt 

Z  I  e  N  e  ?I 

k«0  v=o 


*k 
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1  N-l 


N6 


Jk 


e 


V 


so  that  Z.(t)  consists  only  of  the  elementary  harmonic 
3  2£i 

oscillation  at  frequency  W  .  This  may  also  be  seen 
from  the  filter  function  corresponding  to  Zj<t),  which  is 

1  N-l  1  e-lvu 

*.<“>  -jr  I  •  • 

“  *  v*0 


1  N-l 


But  x(t)  only  contains  oscillations  at  frequencies 
2IIk 

“k  “  $""  »  K»0,...,N-1  and  gj(wlc)  ■ /N  6jk. 


The  representation  (ill)  In  this  case  yields 


x(t) 


N-l 

l 

J*0 


X 


J 


which  is  identical  with  the  spectral  representation  (2.2). 
On  the  other  hand,  since  x(t  +  N)  -  x(t),  the  process 
x(t)  consists  of  only  a  finite  number  of  distinct  random 
variables.  Thus,  considering  only  the  distinct  x(t)'s,  the 


27 


representation  (2.2)  is  identical  with  the  Karhunen  rep» 
reaentation  with  Z,  *  x.. 


Since  an  arbitrary  process  may  be  approximated  by 
a  periodic  one  by  allowing  the  period  to  tend  to  i-nfinity, 
it  seems  reasonable  to  inquire  whether  some  of  the  pro¬ 
perties  exhibited  by  the  principal  component  processes  for 
the  periodic  case  might  not  hold  in  some  limiting  sense 
for  arbitrary  processes.  For  instance,  the  form  of  the 
eigenvalues  cf  the  circular  matrix 


(N-l)  N-l  1 ^  k 

•  I  (e  )  r(k) 
k*0 

might  surest  that  the  set  {1  eigenvalues  of 

v  v*° 

the  covariance  matrix  of  a  stationary  process  might  in 
some  way  approzimate  its  spectral  density  function. 

These  ideas  are  made  more  precise  using  the  theory 
of  Toeplltz  forms  developed  by  Grenander  and  Szego  [  5  ]. 
Definition:  Let  f(x)  be  a  real  valued  function  defined  on 


n 


.(n) 


[-11,  n]  such  that  /  I f(x) |  dx  exists,  and  let  c',“/  be 

-n 

the  matrix  (cy  _  ) ;  u,  v»0,...,n,  where 


n 

/ 

-n 


-ikx 

e 


k 


f ( x)dx. 
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C  ) 

Then  c  1  will  be  called  the  Toeplitz  matrix  of  order  n 
associated  with  the  function  ?(x).  Similarly,  if  P(x)  ia 
monotone  nondecreasing  in  On, n],  the  Toeplitz  matrix 
associated  with  P(x)  is  -  (ctf-  )  where 

1  n  -ikx 

c.  »  ?TP  /  e  dF( x) . 

K  -n 


The  quadratic  form  X  X*  is  called  the  Toeplitz  form 
associated  with  f(x)  or  P(x)  where  X  -  ( x0,x^, . . .  ,xn)  is 
an  arbitrary  complex  vector. 

Proposition;  Let  x(t)  be  a  second  order  stationary  sto¬ 
chastic  process  and  R^n)  the  covariance  matrix 


R(n)  - 


r(0)  . 


r(-n) . 


r(n) 

r(0 ) 


Then  Is  the  Toeplitx  matrix  associated  with  the  spec¬ 

tral  distribution  211  P  (u)  (or  spectral  density  function 
2n  f(di)  if  it  exists). 

Proof;  Bcchner’ s  theorem  for  the  discrete  case  gives 


r(n)  *  /  e^riy  dF  (u)  so  that 

-n 


n 

; 


lnw  d[2n  Px(w)]. 


1 

r(-n)  ■  JIT 


—  u 
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Definition: 


numbers  {ay^ 


For  each  n  consider  the  two 

n).n  .  (n) ,  1  , 

}  and  {by  }  ,  where 

v*0  v*0 


sets  of  real 

there  exists  a 


constant  K  independent  of  n  and  v,  such  that  |av^n^|  <  K, 
|b,/n^|  <  K*  {a  anr  (b  n-*®  are  said  to  be 

1  V  1  V  \j  9 

equally  distributed  in  [~K,K]  if 


lim  . 
n-*-® 


n 

E 

v*0 


tF(a„(n))  -  Fib/10)] 
n  +  1 


0, 


where  F(t)  is  an  arbitrary  continuous  function  on  [-K,K] . 
Theorem:  Let  f(x)  be  a  real  valued  function  such  that 
n 

/  |f(x)|dx  <  ®t  and  let  m  and  M  denote  the  essential 

-n 

lower  and  upper  bounds  (assumed  finite)  respectively  of 
f(x).  If  F(X)  is  any  continuous  function  on  then 


lim 

n-*’® 


F(Xn(n))  +...+  PCX  (n)) 
«  n 


n  +  1 


1  n 

*71 T  /  F(f(x))dx, 

-n 


where  XQ ^ . . .  j>  Xfi  '  '  are  the  eigenvalues  of  the 


<n) 


(2.^ 
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matrix  associated  with  f(x).  For  proof  see  [  5  ]. 

By  considering  the  integral  on  the  right  hand  side  of 

(2.$)  as  a  limit  of  approximating  sums,  the  theorem  shows 

that  the  sets  {A./n^}”  „  and  { f <- TI—  2(v+l)IT  wn  are 

v  v-0  1  “^*2 -  >  >v.0 

equally  distributed  as  n  -*  •»,  and  by  proper  choice  of 
F(X)  it  follows  that 

lim  X  (n)  -  M.  (2.5  ) 

n-M*  u 

These  results  show  that  when  x(t)  is  a  second  order 
stationary  process  with  spectral  density  function  fz(u) , 
the  eigenvalues  . .  ,Xn^n)  of  tend  to  approxi¬ 

mate  2n  fx(“)  at  an  equally  spaced  set  of  points  in 
[-n,n],  and  in  particular, 

lim  Xn(n)  •  max  2n  f  («) 

-n<  &»<n  * 

Pertaining  to  the  asymptotic  behavior  of  eigenvectors ,  it 
Is  shown  in  [  5  ]  that  a  Toepiitz  matrix  can  be  approxi¬ 
mated  according  to  a  certain  norm  by  a  hermitian  matrix 
(with  known  eigenvalues  and  eigenvectors)  in  such  a  way 
that  the  distributions  of  eigenvalues  of  the  two  matrices 
are  asymptotically  equal.  The  mode  of  convergence,  how¬ 
ever,  does  not  seem  to  be  strong  enough  to  guarantee  the 
convergence  of  a  transfer  function  given  by  an  eigenvector 
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of  a  Tceplitz  matrix  to  the  sans  limit  as  that  of  the 
transfer  function  given  by  the  corresponding  eigenvector 
of  the  approximating  matrix.  In  particular,  it  is  shown 
that  an  arbitrary  Toeplitz  matrix  can  be  approximated  by 
a  matrix  with  eigenvectors  given  by  (2.3)-  This  might 
suggest  that  the  principal  component  processes  might,  in 
general,  tend  to  give  a  frequency  decomposition  as  in  the 
case  of  the  periodic  process.  That  this  is  not  always  the 
case  can  be  <ieer.  by  considering  a  process  x(t)  of  the 
opposite  extreme. 

x(t)  A  Discrete  White  Noise  Process 
Let  x(l)  be  such  that  E  x(t)  *  0,  and  E(x(t)  x(t' )} 

«o2  5tt'*  The  8Pectral  density  function  of  this  process 
is 

q2 

fx(A)  ■  2JI  ■>  -n  <  a  <_  n, 

and  the  covariance  matrix 


l  0  V  J 

with  eigenvalues  ■...«An*'n'*  ■  o2  and  corresponding 

eigenvectors  ) , , . . ^ which  may  be  taken  as 


asuis'-xj. 


r 


l 


ip 


( n ) 

given  by  $k  is  hence, 


The  transfer  function 


so  that  each  transfer  function  passes  each  frequency  with 
equal  weighting.  Thus,  while  the  eigenvectors  in  this 
case  are  not  uniquely  determined  (  and  hence  neither  are 
the  principal  component  processes)  for  at  least  one  choice 
of  eigenvectors  the  principal  component  processes  do  not 
provide  a  frequency  decomposition  of  the  original  process. 

It  will  be  seen  in  what  follows  that  for  many 
ctner  common  proteases  certain  of  rhe  associated  principal 
component  processes  bind  tc-  give  meaningful  frequency  de¬ 
compositions  . 


x(t?  A  First,  Order  hcvln.t  Average  Process 


Let  Z(t)  be  a  whitn  noise  process  satisfying 
L  £ ( t )  m  0,  E{  Z  ( t ) , 2  ( t ' ) }  »  and  let 

x(t)  -  Z ( t )  +  pZ(t-l),  whe^e  C  <  |o|  <  1. 


I 

r 
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Then 


*  l+o2  0  o  . 
0  1+p^  0  o 


,<n> 


. .  o] 
.  .  0 


0  ....  0  p  l+p‘ 


The  epectrel  density  function  fx(«>  of  x(t)  is 

1 

rx(e)  •  |  1  +  p2  ♦  pe"*w| 


•  3TT"  (  1  ♦  p2  +  2pcoa«). 


(2.6) 


Prom  the  equation  of  "mechanical  quadrature"  (see  [5  ]) 


l  n  i  n  ,vn  . 

31T  /  *(x)  dx  -  TFT  E  ♦lIT  \ 

-n  v»-N+i 


which  holds  when  ♦(x)  is  a  trigonometric  polynomial  whose 
degree  does  not  exceed  2N-1,  it  can  be  shown  that  the 
eigenvalues  of  R'n^  are  given  by 


(_)  0  ( u+1) 

•  (1  +  pZ)  +  2pcos  n+2 


(2.7) 


with  corresponding  eigenvectors. 


3*1 


a  (n) 

Tv 


{ n  )  1 

■ju 


k(n) 

vn 


,  where 


(n) 


Kvk 


*  2  (k+l)(v-H) 

,  n+2  sin  n+2 


»  v«0,...tn. 

(2.8  ) 


When 


p  >  0,  AQ<n>  >  >...>  and  XQ(n>  (i+p)? 


*  fx(0)  *  mg*  fx(w),  If  p  <  0,  the  subscripts  on  the 
X(n)’s  must  be  reversed  in  order  to  have  the  proper  order¬ 
ing.  After  relabeling 


x 


0 


(n) 


(1  +  o^)  +  2pcos 


(n+l)n  n-*—  o 

n+?  - f  (1  -  p) 


■  f  (ill)  ■  max  f  (uO . 
u> 

For  the  case  n  »  1,  p  >  0, 

2 

■  (l+o2)  ♦  2pcosn/3  >  A  ^  *  (l  +  p2)  +  2pcos?JI, 


A  (1)  _ 

•  * ,  * 

'  142* 

1 

-i45 

► 

z  *  jy  *(0  +  x(t-i). 
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The  transfer  functions  corresponding  ro  Z^^(t)  and 
Z^^(t)  are 


<1)C“>|2  • 1  /7  *yr 

2  1  1 

I*  a)'“>l  *  1  f?  -J? 

Z1 

in  figures  1  and  2 


e  x  |  *  1  *  cosw  and 


»  2 

e-iU|  »  x  -  cosu  as  shown 


Similarly  for  any  finite  n,the  eigenvector  corresponding 
to  the  largest  eigenvalue  yields  a  principal  component 
process  whose  transfer  function  passes  th  largest  portion 
of  fx(w). 


Proposition: 


lim  f 
n-*-®  Z, 


(n) 


(u)  *  0  for  any  fixed  uj^O,  p>0 
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Proof:  r.  P) 

7  in) 

*0 


(k-t-l)Tl 

n+T" 


) 


-iko) 

e 


Kk+pn 

'  "“+7“ 
e 


l(k+l)n  _ 

n+?  1  -ikw 

e  J  e 


I  e 
k-0 


~  “  )(n+P 
e  ' 

- n - 

1  n+7  "  “ 

e  -1 


1 

7TTT 


Nln(u,)  N2n(u) 


For  u)“ui  0 
0 


» 


Dln(w)  and  D2n(u)  ■*  e”^w  -  1  *  0,  while  lN_n(w)l  7  and 

|Nln(w)|  «.  2-  Thus  as  n  -  -,g  (n)(<*>)  -  0  and  f 
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(n) 


(u>) 


f  (u) 
x 


0.  Hence,  the  transfer  function 


0 

-J 

IK  /  x  C  to )  1  ^  acts  like  a  5  function,  <5(ut),  as  n  ■*  ». 
7  v  n  / 

similar  argument  shows  that  when  p  <  0, 

Ik  (n)(w)|2  •*  -  n)  as  n 

zo 


A 


x(t)  A  First  Order  Autoregressive  Process 

Let  Z<t)  be  a  white  noise  process  as  above  and 

x(t)  -  px(t-l)  •  Z(t).  (2.8) 

(2.8)  may  also  be  written 

(1  -  pB)x(t)  ■  Z(t) , 

where  B  represents  the  "backwards  lag  operator" 

(i.e.  Bx( t )  »  x(t-l)),  or 

1 

x(t)  •  1-pfe  Z(t).  (2.9) 

If  x(t)  is  to  be  a  stationary  process  then  [p|  <  lf  in 
which  case  (2.9)  may  formally  be  written  as 

x(t)  -  Z  ov  Bv  Z(t),  or 
v«0 

m 

x(t)  «  Z  0V  Z(t-v). 
v-0 


(2.10) 
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rj*  •< 

'  ‘<*s  V  i-  «  ^ /  } 


>(  n ) 


can  be  seen  to  have  the  form 


R<"> 


a  ^ 


1  P  P 
P  1 


n  n-1 
0  0 


(2.11) 


2  2v 

where  a  *  £  p  .  x(t)  has  spectral  density  function 

v"0 

f(u)  M  “in  f  rhz  I2  "  “k  (1  +  p?  "  2PCOSU))"1 

1-e 

(2.12) 

which  assumes  Its  maximum  at  u>  *  o  when  p  >  0.  While 
explicit  expressions  for  the  eigenvalues  and  eigenvec¬ 
tors  of  R^n ^  are  unfeasible,  something  can  be  said  about 
the  asymptotic  properties  of  the  largest  eigenvalue  and 
corresponding  eigenvector. 

Definition :  A  matrix  A  «  (a^j)  I3  called  positive  if 
ajj  >  0  for  all  i,J . 

Then  when  0  >  0#  is  a  positive  matrix.  This  will 

be  assumed  to  be  the  case  in  what  follows. 

Theorem  ( Perron ) ;  [see  2  p.  278]  If  A  is  a  positive 
matrix,  there  is  a  unique  eigenvalue  X (A)  which  has 
greatest  absolute  value,  is  positive,  and  simple.  Its 
associated  eigenvector  can  be  taken  to  he  positive 
(have  all  positive  elements). 
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I 


Theorem (2 ^Qj.rp-]  v  ,  lim  A  c  exists  and  1?  an 
~  k-»«  UTK)  ]k 

eigenvector  of  A  associated  with  \(k)  for  any  positive 
vector  c.  v  is  unique  up  to  a  scalar  multiple  depend¬ 
ing  on  the  choice  of  c,  but  otherwise,  Independent  of 
the  choice  of  c. 


Lemma  (2.1):  Let  4>n  - 


00 


$r>n 


be  the  positive  eigen¬ 


vector  of  corresponding  co  the  largest  eigenvalue. 

Then  0  *  *00  i  ‘01  „.l  *  *0  „♦!  i— I  *0n 


♦Q0  if  n  is  odd. 


0  *  *00  -  *01  »o  n  !  -  *n  -  ♦„ 


+  1 


7-1 


>..•>  $0n  -  *00  If  n  is  even. 


Proof:  Only  the  case  n  even  will  be  treated  explicitly. 
Positiveness  can  be  assumed  as  a  result  of  Perron's 
theorem.  From  theorem  (2.0)»  if  c  is  chosen  to  be  the 
vector  of  l's,  then  for  any  r 


^0 


<R(n))rc 


«;  ( r  1 

1 


(r) 


where  £  is  the  Jth  row  sum  or  (R^n^)r  or  S 

J  J 


n  (n)  (r-1) 

Jv 


v*0 


l  t),.V"  S..'*  " ' ,  where  is  the  element  in  the 


Jth  row,  vth  column  of  R^n\  The  equality  of 

and  is  obvious  from  the  form  of  and  c.  For 


r  •  1, 

S 


(1)  4 ■  n  4-  4-  .  C 

■1+0+0  +...+  P  <  o. 


1  +  2p  +  p£ 


+.,.+  on_1  <»..<  Sn(1)  •  1  +  2c  ♦  2p2  +...+  2o^ 

1 

>...>  S  (1)  -  S  <*>.  For  r  ■  2, 

n  0  * 


S  * 2 ^  *  S 
0  0 


(1)  +  pS1(1)  +  p2s2(1)  +...+  pnsn 


(1) 


(2) 


■  pS, 


(1) 


+  s. 


while 


so 


;x(1)  +  Ps2^)  +...+  o-V1). 

S1(2)  “  so(2)  “  (1-P)(S1(1)  -  s0(1))  +  (p  -p2)s2{1)+... 

+(pn“^  -  on)Sn(1)  >  0,  and  by  the  same  type  of  argument 

it  can  be  shown  that  S^2'*  *  S^2^  <...<  S  (2)  >... 

2 
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>  S  ^  by  using  only  the  elements  of  and  the 

ordering  of  the  S^^'s.  Hence,  by  induction,  the 
sane  ordering  holds  for  each  power  r  and  the  lemma 
follows  by  taking  the  limit. 


Lemma  2,2:  Under  the  hypothesis  of  lemma  1,  $Q 

7 

•  max  ■*  0  as  n  ■*  •. 

5  0J 

Proof: 


1  P  p2 . o" 

a  (n) 
*00 

P  1  0 . 0 

• 

• 

*n  n  i  n 

*  <n> 

*01 

• 

• 

R(n)  ^(n)  .  02 

p  P 7  ...plp...p2 

• 

• 

;  «") 

0  n 

*  T 

• 

*n  «n"1  a  1 

P  P  i  t  •  *  ¥  * 

a  (n) 

00  J 

(n) 

\n) 


:  (n) 
an 


♦ 


(n) 

00 

(n) 

01 


,  (n) 

*0  n 
2 


V"’  ♦oS"1 


where 


i 


< 1—0 ) ( 1+0 ) 

— rr-vp" 


In  the 


'13 


1+P 

*  1-p  =>  1  ♦  ?o  +  2o‘  ♦  ... 


expression  for  a^n  it  follows  from  lemma  1  that  the 

7* 

coefficient  of  each  positive  power  of  p  is  <  2.  Hence 

a(n>  2 

in  order  for  to  approach  1  ♦  2o  +  2p  +..»  it 


follows  that  for  each  fixed  k,  0  j*  -k  ♦  1*  On  the 

2 

*7 nT“ 

♦o  n 
2 

n  i  \ 

other  hand  since  Z  •  1,  then  ;n'  +  0  as 


v-0 


0  n 
1 


n  «. 

Theorem  2.1:  Let  he  as  in  the  preceeding  lemmas, 

and  let  gQ(n)(a>)  -  Z  ♦n(,n)  e**lv“.  Then  | gn (n) (w) | 2 
u  v-0  Dv  0 

•+  6(u)  as  n  -*•  •*>, 


Proof:  |g./n^ (u) I2  ■  Z  c  eikw,  where  c  -  c  k 

0  k».n  k  k 

-  Z  ^Ov-k  80  that  in  particular  cQ  -  Z 

v«k  va0 


C  ui 


'1.  For  k  < 


(n) 


.1-0  °J 
(n)2 


?  -  ?  ,  <">  *  <"> 

j-k  »J-K 


(n)^  {.n)*  (n),  fn) 

*00  *Ok-l  *0k  v<p0k 


(n) 


r00 


JI4 


+  «  (n)(Ortl  ~  ♦  *,)  <t>n  J$n  -  ) 

Ok+1  Ok+1  01  OnOn  On. 

2  2  T 


*  *0  a^O  n+l  •  *0  n+1_k>  *•••**0  „-lU°  n-l-*o  n-k-r 


*  *0n  <*o„  -  *On-k>-  But  »00  ’  ♦(,„*  *01 


n-l» 


♦  •  • 


•  %  r>  ,  *  *0  l"plles  *0k  (*0k  -  *00> 

7~  7 


~  ^  ill 


♦  S2  >  0.  Since  4>c 


^Qk 

_  A 

*00 

,)  -  ♦, 

that 

the 

above 

k-1 

2 

n/2 

7. 

+  E 

J-o 

o.J 

J-k 

(n) 

-  0, 

then 

0,1-k' 


S.  <  k(Ar  +  0  as  n  - 

1  —  0  r. 


Let  r  be  the  greatest  in- 


li5 


teger  such  that  rk  <  and  let's,  *  min( rk+1 

=  min(rk+2,  £)  »  . . . ,  aR_2  =  min ( (r+1 )k-l ,  2.) .  Then  S? 
n/2  0 

jfk  (Vl  "  *OJ-k*  -  ^Ok  "  +  (*0?k  ”  ^0k)?+*** 

+  ^0  rk"  ^0(r-l)k^  +  ^0  n  "  rk^2  +^0  k+1  “ 

2 

+  ...f(*0  -  *0  (r_1)k+1)2  +<♦„„-  *0  Sl>?-  •  •  • 

2 

+  (*0  2k- 1  “  *0  k-l)2  +***+  (*0  *  *0  rk-i>2 

+  (*0  n  -  *0  a.  .)2*  Now»  each  term  of  <*0k  "  *oo} 

7  *c“1 

*  (*0  2k  -  %k>  +"-*  (*o  rk  '  *0  Cr-l)k>  +  <♦„  n  -*0  rk; 

T 

*  (♦on"  ^00^  non”nePative  so  that  the  sum  cf  squares 

2 

of  these  terms  Is  less  than  or  equal  to  the  square  of  the 
sxm.  The  same  is  true  for  each  of  the  other  k-1  sums  so 
that  S2  k(<J>Q  n  -  ♦qq)2  -*■  0  as  n  •*■  <**.  But  this  implies 

that  for  each  fixed  k,  c  »  c 


•+  1  as  n  -*■  ®.  From  this 


i»6 


it  follows  that  for  any  positive  integer  ‘I,  and  any 
e  >  0,  there  exists  an  integer  *'  such  that  n  >  M 


1  -  c 


(n) 


211+1 


for  k 


0,1,.. . ,N.  Then 


r  c.  <n)  elk“-  t  elk"l  <  I 


k»-M  ^ 


k*-N 


N 

Z 

k“-N 

N 


ck<n)  -  1|  <  e. 


iktjj 


Thus,  if  the  partial  sums  S..(u!)  »  Z  e  of  the 

N  k«-N 


“  iku 

trigonometrical  series  I  e  converge  to  a  general- 

k*~“ 


ized  function  g(u>),  the  transfer  function  |  gQ^n^  (id)  |  2 

n  fn\  ik(i> 

■  £  ckv  e  must  converge  to  the  same  function. 

k»-n 

It  is  shown,  however,  in  Lighthill  [6,  p.  67]  that  the 

OO 

iktu 

trigonometrical  series  E  e  converges  to  the  gen- 

^a«eo 


eralized  function  g(oO  *  211  E  (w  -  2kII),  (a  train  of 

k=-°° 


delta  functions 

[-n,n],  g(w)  is 


lg0<n)(»>l2 


n-*<*> 


.  When  m  is  restricted  to  the  interval 
simply  2TT<i(w),  and  it  then  follows  that 


2n<5U) . 


Now  suppose  o  <  0.  Let  the  covariance 
matrices  and  spectral  density  functions  now  be  denoted 
/  n  \  _ 

by  R  and  Since  f“(u)  •  f  (u  +  n),  it 

follows  from  the  theory  of  Toeplitz  forms  that  the 
eigenvalues  of  R^“  are  the  same  as  those  of  R^n^. 

Theorem  2.2;  Let  gln^“((j)  -  Z  4n^n^_  e"lv“t  where 

v-0 

is  the  vth  element  of  $0^”,  the  eigenvector 

/  n\  (n)- 

corresponding  to  the  largest  eigenvalue  XQV of  R 

Then  60^n^”(“)  ♦  0  as  n  •*  •  for  any  fixed  u  such  that 
|w|  II. 


Proof: 

that  4 


By  direct  multiplication, 
oi"*"  *  (-l)”  ♦0^n)  so  that 

(.)  -  ?  ♦0<")  e”lvu 

v»0 


it  may  he  verified 


-  n(n)  e-i(w*n)v 

v-0  0v 


whenever  (u  t  n)  ?  0 
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These  same  results  can  be  obtained  to  show 
the  behavior  of  the  first  principal  component  process 
of  certain  higher  order  moving  average  and  autoregress¬ 
ive  processes.  For  Instance,  consider  the  second  order 
moving  average  process  x(t)  *  7. (t )  +  o^ZCt  -  1) 

+  a2Z(t  -  2),  where  Z(t)  is  a  white  noise  process, 
0<a.  <l,0<a2<l.  x(t)  has  spectral  density 
function  f* x C *jj )  a  — (1  +  +  a2  +  2(a^  4-  a^c^cosu 

+  2a2cos2w)  which  attains  its  maximum  at  u  »  0.  The 
covariance  matrix  in  this  case  is 


(  2  2 


i  +  Vra2 

°l+0ll“2 

2  2 

“l  +  c‘;a 

1+3^+oc., 

“2 

a^+Ci «2 

0 

■ 

• 

e 

• 

0  .  .  . 

III! 

«2  0 . >0 

a  2  0 . 0 

2  2 

l+c-^+«2  ai+°]°2  *2  0...G 


a  a  ♦a,a_ 
2  1  1  2 


3 


2J 


The  results  of  Perron's  theorem  and  its  corollary  can 


HO 


be  shown  (Frobenius'  theorem  [J'D  to  hold  for  an  Irre¬ 
ducible  matrix  A  *  (a^),  where  A  is  called  irreducible 
if  aij  >  0  and  if  whenever  apq  *  0  for  p  i<  q,  there  ex¬ 
ists  some  set  of  indices  l^,...tir  such  that  the  indices 


p,i-  ,...,i  ,q  are  all  different  and  api  /  0,...,a  ^ 

1  1r-l 


*0 


_  P  0 .  It  can  easily  be  verified  that  R 
rq 


<n) 


is  irre¬ 


ducible,  for  example,  n+^  *  0  but  a13  *  °*  a35  *  °» 


•  •  •  3  *  0. 

n-1  n+1 


Lemma  2.1  holds  by  exactly  the  same  proof  as  in 

the  autoregressive  case.  To  show  that  lemma  2.2  holds, 

.  .  D(n)  (n) 
consider  R 


fadyO . o’ 

« 

♦o 

8  u  8  y  0  .  .  .  « . .  0 

♦l 

y  B  a  B  y  0 . . 0 

0  y  8  a  8  y  0 . p 

• 

2 

• 

• 

• 

0 . 0y3o6y3....0 

• 

$n 

• 

• 

■ 

.  4 

♦o 

2  2 

and  a  s  1  +  a  +  a„  .  3  =  a  +aia0.  v  =  a,  .  Since 

1  d  1  i  i  1 

<$.  Is  an  eigenvector,  =  a  =  ...  «*  a 


Xn  n'  and  as  n  ■*  A  -»  f  (0)  *  a  +  2B  +  2y. 


The  expression  for  a  '  ‘  thus  shows  that  as  n  ■*  «, 


<J>  /♦  -*■  1  and  <$n  / <J>  -»  1.  Working  backwards  then, 

2”1  7  2~2  ? 

It  follows  that  for  any  fixed  k,  $n  /4>n  •*  1  as  n  -*•  * 

7-k  2 

and  the  lemma  follows.  Theorem  2.1  then  holds  exactly 
as  before.  If  <  0,  o2  >  0,  then  nl  +  <  0  and 

the  spectral  density  function  now  denoted  by  f“(u) 
assumes  Its  maximum  at  A  -  ±H.  f~(u>)  *  f  (u>  +■  H)  so 

X  X 

that  the  eigenvalues  of  are  the  same  as  those  of 

except  with  different  ordering,  where  R^n^“  is 
the  covariance  matrix  in  the  case  with  <  0 .  R^n)~ 

can  be  derived  from  R^n'1  by  changing  the  algebraic  sign 
of  every  other  element  and,  hence,  direct  multiplication 


will  3how  that  A 


,  where  $ 


.  (n)- 
On 
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\)  (  n  ' 

*  (-1)  $  “  so  that  theorem  2,2  holds. 

0  V 

Mixed  Autoregress Ive  Moving  Average  Process 

Let  x(t)  be  a  stationary  process  of  the  form 
x ( t )  -  px(t-l)  »  Z(t)  ♦  aZ(t-l).  (2.13) 


This  can  be  formally  written  as, 


(1  -  pB)x(t)  -  (1  +  aB)Z(t) 

or, 

(1  +aB) 

x ( t )  <*■  (1  -pS)  Z(t),  where  B  denotes  (2.14) 

the  backwards  lag  operator.  Prom  (2.14)  if  |p|  <  1,  it 
follows  that  x(t)  can  be  expressed  as  the  infinite  mov¬ 
ing  avera re  orocess 

x( t )  *  Z(t)  +  (P  +  a)Z(t-l)  t  P(P  ♦  «)Z(t-2)  +... 
from  which  it  follows  that 


r(0 ) 


1  +  2pa 


r  - 


» 


r(n)  ■  on~1fc»  ota2  +  pa  ♦  1)1  -  pn  1Y  • 

i .  o5  Ho 


so  that , 


A 


x  2  n  •  1 

6  Y  PY  P  Y  .  .  .0 jV 

Y  5  Y  DY  e  •  .  pn_^Y 


r^F2 


If  p,a  >  Of  is  a  positive  matrix  and  if 


«.  (1)  -  (1) 
bo  *  si 


S  denote  its  row  sums,  then 


*0a>  <  S1U)  3n(:>  SnU)  ■  5oa>-  S0<2> 

2 

-  «s0(1)  +  ySjl(1)  +  pyS2(1)  +...+  p 7  ysn(1)  +.♦. 

+  Pn-1«Sn(:L),  S1(2)  -  YS0(1)  +  6S1(1)  +  YS2<l)  +... 
n  ] 

+  ysJ1)  pn_2SS  y  <  5  since  it  is  al¬ 


ways  true  that  |r(t)|  <  r(0)  and  y  is  real.  Hence,  it 
f  2 )  (2) 

follows  that  Sj,  '  >  Sg  ,  and  in  a  similar  way  Sn'  ■ 

2 


>  ^  (2)  >  „  5  (2)  v  (2) 

9  9  *->n  9  ctC,  . 

H-i  u 


which  implies  that 


lemma  2.1  holds.  The  spectral  density  function  of  x(t) 


f  (u)  *  5 iT 


1  +  ae 


1  -  pe 


1  1  +  a  +  2acosio 

-  tit  r  "+  p^  -  2pcosw 
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For  fixed  n,  the  row 


r;.  (  1  ) 


n 

T 


7” 


(6  ♦  2y  +  ?PY  +...+  ?0  Y)  “  ? 

~  0 


1-0 


7  [  1  +  2 pa  +  or  +  2  ( a  +  o(  a *  pa  +  1 ) ) 


n  i 

2  7”1  o 

♦  p(a  +  p(a  +  pa  +•  1))  +  ...+  2p  (a  ♦  p(a^  +  pa  +  1))] 


i  ^  B-i 

-  1  _  p?  [(1  t  a)2  +  2o(l  +  a)2  +  ...+  2p2  (1  +  a)2 

n 

7  ?  2^1 

+  2p~  (a  ♦  a  +  1)  +  2p  a].  As  n  -*•  <*>  thl3  becomes 


i  ,  ,  i  ?  a  +  p) 

!  r?  u  +  a)  (rf P-  x>  * itt?  u+a)  tt-^pt 


(1  ♦  a}2 


♦  a)‘ 

*  Tl  -  p ; d  *  2 n CO).  The  same  argument  as  In  the  auto¬ 
regressive  case  then  shows  that  lemma  2.2  and  hence 
theorem  2.1  also  hold  in  this  case. 

If  both  p  <  0  and  a  <  0  then  6  >  0,  while  y  <  0» 
f“"(w)  *  f  (n  +  w)  has  Its  maximum  at  u  »  ±Tt; 

X  *  o 


n>-  1 


►  ( 

00 


4R1- 


,  where  ■  ( -1 ) V<*0^n ^  and  hence 


theorem  2.2  holds 


CHAPTER  III 

PROCFSSES  WITH  NONCONSTANT  MEANS 

Let  y(t)  be  a  real  valued  second  order  sta¬ 
tionary  process  with  mean  zero,  and  let  x(t)  *  m(t) 

+  y(t),  where  m(t)  is  a  deterministic  but  Unknown  func¬ 
tion.  In  this  chapter  the  covariances  considered  will 
be  expected  sample  covariances  where  the  sample  covari¬ 
ances  are  computed  as  if  m(t)  were  constant.  Thus,  each 
covariance  will  consist  of  a  part  due  to  m(t)  and  a  part 
due  to  y(t)  as  follows. 

Let  x ( 1 ) ,  x( 2) , . . . ,  x(n  +  k)  be  a  sample  from 
the  process  x(t)  and  define 

x(r)  *  n  E  x(t  -  r)  =  m^  +  y^,  r»0*.*,,k 
t-k+1 

(3.1) 

The  sample  covariance  between  x(t)  and  x(t  -  t)  can 
then  be  written  as 

1  n+k  ,  , 

S(t)  *  n  E  (x(t)  -  x^°')(x(t  -  t)  -  ) 

t»k+l 


55 


56 


a  n  E  (m(t)  +  y(t)  -  -  y^°^)(m(t  -  t) 

t*k+l 


+  y(t  -  t)  -  m(T)  -  y(T)) 


n+k  , < r) 

I  m(t)m(t  -  t)  - 
t»k+l 


+  n  E  m(t)[y(t  -  t)  -  y^T^] 

t-k+i 

i  n+k 

+  n  E  m(t  -  x)[y(t)  -  y^0'] 

t«k+l 

+  n  E  y(t)y(t  -  t)  -  y^°^y^T^ 
t-k+1 


Since  y(t)  is  stationary  then  (approximately)  ES(t) 


1  n+k  frn  (t)  j. 

•  n  E  m(t)m(t  -  x)  -  m'  mv  *  r  (t  } ,  C}*2) 

t*k+l  7 


If  y(t)  is  a  white  noise  process,  then  the  equation  be* 
comes 


1  n+k  w _ \ 

E  S(t)  -  n  E  m(t)m(t  -  t)  -  mtO'nr  ’  + 
t*k+l 


tO 


(3.3) 


I 
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x(t)  A  Linsar  Trend  Plus  White  Noise 


Let  x(t)  =  m(t)  +  Z(t),  where  Z(t)  la  a 

white  noise  process,  EZ,(t)  ■  0,  E(Z{t)  Z(t  -  t)) 

2 

■  a 2  ®ot»  m(t)  *  a  +  bt  where  a  and  b  are  constants, 

Then  m(t)  m(t  -  x)  *  (a  +  bt)(a  +  b(t  -  x)) 

*  (a  +  bt)((a  -  bx)  +  bt)  «  aa  +  (ab  +  a  b)t  +  b2t2 

X  T 

1  n+k  .  .  .  n+V 

^  E  m(t)  m(t  -  x)  *  aa  +  b^a  ax^  E  t 

n  t-k+1  n  t-k+1 

.2  n+k  O  f  \ 

+  £  E  t2,  while  m(0'm(T) 
n  t-k+1 


n+k 


n+k 


Z  (a  +  bt)  —  E  (a  +  b(t  -  t)) 


t-k+1 


n 


t-k+1 


.  n+k  .  n+k 

(a  +  g.  E  c)(aT  +  -  e  t) 

t-k+1  T  n  t-k+1 


b(a  +  a  )  n+k  b2r  n+k  2 
*  aa  +  t  E  t  +  — E  t] 

n  t-k+1  n  t-k+1 


2  n+k 


n+k 


.C-  '  r.  ry  ^  II'A 

By  (3-3)  ES_  -  2.  I  %£  -  H  T.  t] 

n  t-k+1  n  t-k+1 


2  X  2X 

+  o  6  , 

Z  tO 


Hence,  the  expected  covariance  matrix  ER^n^ 
form 


5-8 


ha5  thfi 


ER(n>  a 


o  2+a  a, . a 

a  o^+a  a  •  •  .  o 


where 


a 


a  c2+ci  j 


a  «  the  first  term  above.  This  is  a  positive  circular, 
matrix,  and  hence  for  a  given  n,  ER^n^  has  eigenvalues 

X^(n)  «  a2  +  a  +  {  E  )  o,  where  rv  is  a  root  of 


xn+1-  1-0.  When  r  »  1,  the  corresponding  eigenvalue 
is  Xq ^ *  o2  +  (n  ♦  l)a,  while  if  rv  ?  1,  Xv^  ■  a2. 


The  eigenvector  corresponding  to  XQ^n^  is 


/  \  2 

whose  transfer  function  Is  (g^  n;(w)| 

1 

■  jrz~i 


n 

E  e 
k«0 


-ikw 


But, 
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I  -  e“lM(n+l)  ^ 

j  ^  giw  "  n  J  *  0  for  any  fixed  u  j*  0. 

On  the  other  hand  since  m(t)  is  non-periodic  (a  linear 
trend  in  this  case)  it  can  be  considered  as  having  the 
Fourier  representation 

n 

m(t)  •  I  eltu  du(u>), 

-n 


where  du(w)  *  0  for  a  f  0.  As  a  result,  the  (sample) 
power  spectrum  of  x(t)  consists  of  the  spectrum  of  y(t) 
plus  a  Jump  at  #  *  0  due  to  m(t).  The  form  of  the 
above  transfer  function  shows  that  the  first  principal 
component  tends  to  remove  this  zero  frequency  or  trend 
portion  of  x(t) 


x(t)  A  Periodic  Function  Plus  White  Noise 


Consider  the  process  x(t)  *  cos(^>t)  *  Z(t), 
where  Z(t)  is  a  white  noise  process.  For  convenience, 
it  will  be  assumed  that  both  n  (the  sample  size)  and 
N  +  1  (the  order  of  the  covariance  matrix)  are  always 
taken  to  be  divisible  by  four.  Then, 


i<c)  -  i  T 

n  t-k+l 


eos(2t)  *  0, 


3|t->  3(*»*  3|H* 
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£m(t)m(t)  ■  L  (0  +  1  +  ...)  »  1/2, 


T.m(  t)m(t  -  1)  -  0, 


Em(t)m(t  -  2)  -  ±  (0  -  1  'v  0  ...)  •  -1/2 


-  3)  “  0,  etc. 


Hence,  the  expected  covariance  matrix  1b  of  the  form 


E  R<n> 


cjz24-1/2  0  -1/2  0  ...  1/2  0  -1/2  o' 

0  az?*1/2  0  . -1/2 


0 


Qa^+l/Zj 


This,  again,  la  a  circular  matrix  with  eigenvalues 


xv  *  (o^2  +  1/2)  +  1/2  ?  (-l)^rv2^,  where  rv  is 

J 

a  root  of  xN+1  -  1  *  0.  The  expression  for  assumes 
its  maximum  value  when  (ry)2^  ■  (-1)>J  or  when  rv  *  ±i. 


Since  ru  can  be  expressed  in  the  form  r  ■  e 


v 

i  2nv 
TFT 


v  -  0 , 1 , . . . ,N.  X(max.)  is  attained  for  v  »  — and 

v  *  1^ N+l) ^  u 3 1 n g  the  same  notation  as  in  Chapter  II, 
two  orthogonal  eigenvectors  corresponding  to  the  double 
root  X  (max.)  are  given  by 
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.(N) 


T' 


N+1 


1 

1 

in 

i  31 

•-* 

e 

• 

• 

• 

an<i  ^  "v/tM 

• 

• 

• 

iRN 
e  ? 

m 

1  3n  N 

e  ~ 

L  » 

Let  be  an  arbitrary  linear  combination 


(N) 

)' 

to  i3 


o*N)  *  b*o  The  transfer  function  corresponding 


ig(N)(«)r 


N 

I  ^ 

k*0 


Jim 

N 

__  l 

{n+l  k-o 


ink  b 
e  “  e 

>Jn+T 


1  3  E  k  ~ik<D .  2 
”  )e  I 


N  i(£~ u)k 

where  g'N'<w)  *  — E  e 


b  i(3£  -»)k 

+  —  Te  ^  ■*  0 


~  "B5S 

4N+1 


with  N  for  any  value  of  w  different  from  11/2  or  3H/2. 

For  t  *  0,1,...,  cos(5-)t  *  cos ( ^— •) t ,  so  that  any  princi¬ 
pal  component  formed  from  an  eigenvector  corresponding 
to  A(max.)  will  tend  to  filter  out  the  deterministic 
function  m(t) . 


APPENDIX 

Figures  1  and  2  of  chapter  II  showed  the 
transfer  functions  corresponding  to  the  two  eigenvectors 
of  the  2X2,  (n  «*  1)  covariance  matrix  of  the  ntoving 
average  process  x(t)  -  Z(t)  +  .9Z(t  -  1).  Figures  3 
and  4  show  how  these  transfer  functions  change  as  n 
increases.  Figure  3  shows  the  transfer  functions  corre¬ 
sponding  to  the  eigenvectors  when  n  •  4, 

while  figure  4  shows  the  transfer  functions  corresponding 
to  <f>0,  for  n  «  6. 
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Figure  5  shows  an  example  of  the  behavior 
of  an  autoregressive  process  x(t)  +  .Sx(t  -  1)  *  Z(i) 
together  with  its  first  and  fifth  principal  component 
processes  for  the  case  n  «  4.  The  series  Z(t)  consists 
of  a  series  of  random  numbers  taken  from  the  Rand  table 
of  Normal  Random  Deviates.  Prom  this  series,  the  series 
y(t)  was  constructed.  The  principal  component  processes 
were  constructed  by  numerically  determining  the  eigen¬ 
vectors  of  the  theoretical  covariance  matrix  of  y(t) 
and  applying  the  appropriate  filters  to  y(t).  The  first 
principal  component  process  can  be  seen  to  consist  pri¬ 
marily  of  oscillations  with  period  two,  while  the  fifth 
principal  component  process  contains  oscillations  of  much 
lower  frequencies . 


9  V 
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Figure  6  shows  a  sample  cf  the  process  x(t) 

*  "TS"  t  +  5cos^-t  +  y(t),  where  y(t)  +  ,9y(t  -  1)  *  Z(t), 
Z(t)  a  white  noise  process.  Also  shown  are  the  first 
and  second  principal  component  processes  for  n  ■  l8. 

The  process  x(t)  was  constructed  by  taking  as  Z(t),  two 
hundred  numbers  from  the  Rand  Table  of  Random  Normal 
Deviates,  constructing  y(t),  and  combining  y(t)  with 
_^-t  and  5cos^.t.  The  data  was  keypunched  and  the  prin¬ 
cipal  components  determined  (from  the  sample  co/ariance 
matrix)  using  the  BMDOIM  principal  component  program. 

This  process  was  selected  as  an  example  since  it  might 
resemble  an  unseasonally  adjusted  economic  series. 

The  first  principal  component  contains  the 
linear  trend  plus  some  low  frequency  oscillations,  while 
the  second  principal  component  seems  to  consist  almost 
entirely  of  oscillations  with  period  four  corresponding 
to  the  cosine  term.  The  third  principal  component  (not 
shown)  also  consists  primarily  of  oscillations  with 
period  four  as  would  be  expected  from  the  results  of 
chapter  III,  while  the  fourth  (also  not  shown)  component 
contains  oscillations  of  period  two  from  the  autoregress¬ 


ive  process. 


Figure  6 
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Figure  7  shows  a  series  x(t)  *  logft  (the 
total  U.S.  Personal  Income)  ,  a  quarterly,  seasonally 
adjusted  series  from  "Survey  of  Current  Business", 
from  first  quarter  1958  through  fourth  quarter  1968. 
Shown  with  x(t)  is  its  first  principal  component  process 
multiplied  by  a  scale  factor  so  that  it  fits  the  ori¬ 
ginal  series.  These  principal  components  were  also  com¬ 
puted  using  the  BMDOIM  program.  In  this  example,  n  was 
fchbsen  to  be  equal  to  six.  Figure  8  shows  the  second 
principal  component  on  a  greatly  magnified  scale. 


Figure  7 


Figure  8 
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Tb“  pr  ix.’iVry  -ourooec  o 4  this  d.isaox'tation  .;  s  T 5. : *  'C 1  i yx i.'*  1  be 
properties  of  the  principal  components  ox'  a  finite  sot  of  random  variables 
comprising  a  part  of  a  discrete  thus  series.  In  tbs  xL^t  pliapter,  the 
covariance  structure  between  sv  set  of  r?."iU's  va.r lob.'.cs  yf  >  *  *  •  }  r; 

which  yields  the  result  that  the  first  fc(  <  p)  principal  exponents  of 

provide  e  Vetter  prcdj.ct.or  of  y  (in  the  se?ise  of  corrected  squared  error)  tlsan 
do  MJ'  h  O’’  tb»  tn..r,^v. vv.':.  is  oveni  r.ed . 


In  the  rer-ai:ii r.r.  chapters .  principal  ecr.ponont  processes  vliich  are  linear, 
*.  :'l .  Z-r>  ../o'*  «*';.••  y  -fv)  ir  a  random  process:  aid  n 

cOiHtjLnctvion^  o*  aIv/,  .*/  *"*  '  .  ..  .  .  , 

m  or.  sroi.irt-.ry  pssLui  r  ^nt^er.  ^-.tklmeu  *n-.i  wcu  properties 

in  oe.i;uii  uf  b.vb  ?a»u..*..uv  e-ufo.-t.  It  Vnat  vl'.e-n  x(t)  is  a 

Roving  average  process,  ass  autoregressive  pioces z ,  or  a  rruxAd  moving  average 
....  j,, ,,,c  ,  jy.i  pro'-rcr iho  first  principal  component  process  tehds  ;.ns  r.  *  -*) 
to  in  ori’r  the  freaw.-ticy  at  •■bice  the  st- .‘-'tr  i  density  of  x(tj  obtains 

j t - ' inaxiaum  value.  It  ir  chovr,  Moreover,  that  when  the  process  x(t)  contains 
de" -m' Lilstf--  c«x-.ooner.t.i  .inch  os  a  trend  or  a  periodic  component,  certain  of 

•'*  c^-.v -vs-.tt.  r.m«.uc?.f-s  tend  to  model  those  deterministic  components. 
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